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Considerations for Making Changes to SIMR Baseline and Targets 


Purpose and Intended Audience 


Overview 


This paper provides Part C and Part B state staff an 
opportunity to learn about considerations that could 
support changing their State-Identified Measurable Result 
(SIMR) baselines or targets. Since states will be working 

on data quality for their SIMR, it is possible that, as data 
quality improves, the data for baselines will change, which 
could lead to revising the targets. 


Introduction 


This paper addresses four central questions: 
1. What are baseline data? 

2. Why change a SIMR baseline? 

3. What isa target? 

4. Why change SIMR targets? 


Important Note: This paper addresses 
considerations for changing baseline and 
targets—states should always discuss proposed 
changes with their OSEP State Lead. 


A State Systemic Improvement Plan (SSIP) is a multi- 
year achievable plan that will increase the capacity of 
early intervention service programs or local education 
agencies to implement, scale up, and sustain evidence- 
based practices to improve outcomes for children and 
youth with disabilities and their families. Under Results 
Driven Accountability, OSEP has placed an increased 
emphasis on improving results for children and youth 
with disabilities. States are required to focus ona 
particular SIMR. The SIMR must be a child, youth, or family 
outcome and cannot be a process outcome. The SIMR 
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might be, for example, a measure of school readiness, 

of achievement, of graduation rates, or of college or career 
readiness. The measure must include either (a) all children 
or youth with disabilities in the state or (b) a subset of 
these children or youth, with an explanation about how 
improving the result for that subset would improve that 
result on a statewide basis. OSEP expects stakeholders 

to be involved in all three phases of SSIP, which includes 
the selection of the SIMR, target setting, and revisions to 
baselines and targets. 
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What are Baseline Data? 


In the context of the SSIP, baseline data provide 
information about the performance of the identified 
group of children or youth before the implementation 


of SSIP strategies. Of most importance will be the baseline 


value of the SIMR. These values can be aggregated 
from the child or youth level to the school, district, local 
agency, or state level. Examples of state SIMRs are early 
childhood outcomes, family outcomes, graduation, and 
postsecondary outcomes. See Appendix A for list of 
state SIMRs. 


Establishing a baseline is important for the SSIP process 
because it serves as a point of reference, allowing 

the demonstration of change over time. When a state 
compares baseline data to data collected at later points 
in time, it will need to determine if the state is making 
sufficient progress toward its targets. Baseline data are 
used to establish targets and the amount of growth that 
is expected each year. The selection of an appropriate 
assessment or data collection tool, the proper use of the 
assessment or data collection tool, and the recording of 
the data collected from these tools are essential for the 
creation of a valid SIMR with a meaningful baseline. 


Once the data have been selected to use for the SIMR, 


they need to be analyzed. The following questions should 


be asked about the baseline data: 


e Are the data of high quality? Are they reliable and valid? 
e If necessary, are there plans to improve the data quality? 


If so, how? 
e How are the improvements in the quality of the data 


likely to affect the baseline and targets? Is the baseline 


likely to go up or down based on improving the data 
quality? 

e What do the data show about children or youth 
statewide? 


e What do the data show about children or youth by race, 


ethnicity, special education, general education, local 
district/program, etc.? 
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e Are there patterns in the data (i.e., are there subgroups 
that have a mean that is higher, lower, or the same 
as the state)? How do subgroups compare with 
each other? 


e Will the differences affect the determination of targets? 


Asking these questions will not only help a state 
to identify appropriate baselines and targets, but 
will also help the state to devise the best coherent 
improvement strategies. 


Why Change a SIMR Baseline? 


Sometimes a state must change the baseline that it 
provided to OSEP in its Phase | SSIP submission. The 
primary reason to revise a SIMR baseline is the suspect 
quality of the data used to set the original baseline. 
Here are some common reasons to question the quality 
of available data: 


1. The baseline data are not representative. That is, 
significant proportions of the children or youth who 
should have been included in outcomes data are 
missing from the baseline data analysis, and those who 
were included in the data analysis differ in important 
ways from the targeted children or youth. If, for 
example, youth from large urban areas or children 
whose home language is not English were not included 
in the original baseline data analysis, the baseline 
may not accurately represent the state’s children or 
youth. When more representative and accurate data 
are collected, the state might consider resetting its 
baseline using the better data. Note that SIMR data 
can be collected by sampling: indeed, a representative 
sample with a high response rate can be more valid 
than an attempted census with a low response rate 
or a response rate that varies by subgroup. 


2. State or local districts/programs have out-of- 
range data. If a state’s outcome data are significantly 
different from the national data, or local districts/ 
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programs within a state have outcome data significantly 3. Changing data collection tools or methods. When 

different from the state data, this may indicate that states change the tool or method for collecting child 

some localities legitimately differ in important ways or student outcomes data, it is often because they 

from other localities. It may, however, be a red flag for were not satisfied with the quality and accuracy of 

inaccurate data. This could be due to one or more of the data resulting from the original methods. For 

the following situations. Part B, the national movement toward Common Core 

a. Problems with data collection methods. Standards has resulted in many new curricula and 
If personnel collecting the outcomes data do not many new assessments. When the new data collection 
use the appropriate data collection methods, their methods are implemented successfully with fidelity, the 
results will be inaccurate. If, for example, early outcomes data may be higher or lower than the original 
childhood teams are using the Child Outcomes baseline. States in this situation may want to revise their 
Summary Process! without understanding the baseline using the data from the new tools/methods. 
rating scale, the ratings may not accurately reflect 4. The state used data from research on evidence- 


children’s functioning. If improvement activities, 
such as professional development, successfully 
improve the data collection methods, the more 
accurate data could be used to revise the baseline. 


b. Problems with data reporting or data system 
errors. Errors in data reporting and data entry 
into data systems may also cause inaccuracies 
in baseline data. If improvement activities, like 
building data system error checks, or professional 
development in data reporting, result in more 
accurate data, a state could propose to reestablish 
the baseline with the more accurate data. 


c. Problems with the particular data collection 
tool. Unusual data patterns may be related to 
problems with the particular data collection tool 
that the state used to measure child outcomes. For 
example, states that use one particular assessment 
tool statewide to measure outcomes may find that 
one or more of the three early childhood outcome 
areas are not measured well by the selected tool. 
An assessment tool may have too few items in an 
outcome to show differences among children or 
youth, or an assessment tool may not cover the 
intended constructs in the outcome area. 


based practices to set the baseline, since it did 

not have its own data. If a state did not have its own 
baseline data it may have used baseline data from 
research. Once the state has collected its own data, 

it can determine the baseline, and if it is different from 
the evidence-based study, the state may propose to use 
its own data to set the baseline. 


Take home message: If the data quality 
improvement activities in the SSIP have 
successfully improved the quality of the outcome 
data, a state should consider using the improved/ 
more accurate data to reset its baseline. 

Caution: If improvement activities have also 
been implemented to improve provider practices, 
and thus improve outcomes, the state will need 
to determine whether any change in outcomes 

is from more accurate data or from improved 
practices. This can be challenging to do, and states 
are encouraged to contact one of the OSEP TA 
Centers for additional assistance. 


1 For more information about the Child Outcomes Summary Process please 
see http://ectacenter.org/eco/pages/outcomes.asp#COSProcessModule. 
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What is a Target? 


In the SSIP context, a target is the value of the SIMR that 

is expected at the end of each reporting period. The 

value of the baseline is one of the factors the state should 
consider in setting SIMR targets that are both rigorous and 
attainable for their children or youth. Baselines and targets 
enable administrators and stakeholders to determine the 
amount of progress that children or youth should make 
during the time period being examined. 


Targets orient stakeholders to the goal to be 
accomplished and motivate states to do their best to 
meet or exceed the targets. Targets help to establish 
expectations for staff, implementing partners, and key 
stakeholders. Once the SSIP activities have started, targets 
serve as guides for monitoring progress and determining 
if progress is on schedule. Finally, targets promote 
transparency and accountability by making available 
information on whether the intended results have been 
achieved over time. 


When setting targets the following should be considered: 


e Targets are best based on the first year of quality data 
collected, not necessarily the first year of data. 


e If the targets are developed from data from pilot 
programs, scaling-up of implementation must be 
considered. 

e If selecting subgroups, consider using the baselines 
of the individual groups to set their targets. 


Why Change SIMR Targets? 


Three situations that may warrant changes in targets are 
described below. 


1. The state’s baseline is revised to reflect more 
accurate data. When a state revises its baseline, it 
will almost always need to adjust its targets to reflect 
growth from the new baseline. The methods the state 
used to project the initial targets might be used again, 
but in relation to the new baseline. For example, if a 
state projected targets that were a 5-point increase in 
the percentage of third-grade students with disabilities 
achieving at the Basic level or better in reading (e.g., the 
percentage of students would increase from 35 percent 
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at baseline to 40 percent by the end of the SSIP), the 
state may choose to recalculate the increase from the 
revised baseline (e.g., the accurate baseline is changed 
to 30 percent, so a 5-point increase would result in 

a new target of 35 percent by the end of the SSIP 
implementation). 


Data from early implementers is not as expected. 
If a state is phasing in its improvements in various 
areas in the state, the data from the first cohort(s) 
may be informative about the appropriateness of 

the state's targets. The local districts/programs first 
implementing the improvement activities will provide 
the state data about the level of achievement that 
occurs when the improvements are implemented. 

A state may want to compare the actual performance 
of children or youth in the first cohort(s) to the state's 
projected targets. If the gains are not as expected, 
either higher or lower, a state might propose to reset its 
targets, aligning its projections with the data from the 
earliest implementers. 


. The state used data from research on evidence- 


based practices to set targets, since it did not have 
its own data. If a state did not have its own baseline 
data to set targets, it may have used baseline data from 
research to establish its targets. Once the state has its 
own baseline and if it is different from the evidence- 
based study data, the state may propose to use its 

own data to project revised targets. (Alternatively, the 
state also should consider whether adjustments in 
implementation of improvement activities would yield 
better results.) 


Take home message: Resetting baseline and 
targets is most often a result of a state having 
better information/data than it had when 

it initially set its baseline and projected its 
targets. The state should use this information 

to thoughtfully change its targets when needed. 
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Summary 


Although Part C and Part B state staff submitted SIMR 


baselines and targets for SSIP Phase I, there may be a need 


for Part C and Part B staff to propose to OSEP reasons for 
revising. Since states will be working on data quality for 
their SIMR, it is possible that, as data quality improves, 
baselines will change, which could lead to revising 

the targets. 


Baselines and targets are important because when used 
together they provide an opportunity to determine if 
states are making progress toward their outcomes. The 
primary reasons for a state to request a change to the 
SIMR baselines are due to data quality. States are dealing 
with issues such as representativeness of the data; data 
collection methods, data reporting or data system errors, 
data collection tools; changing data collection tools or 
methods; and not having their own data to report in the 
SSIP Phase I. 
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The primary reasons for a state to request a change to the 
SIMR targets are due to the state’s baseline being revised 
to reflect more accurate data; unexpected data from early 
implementers; and not having its own data to report in 
the SSIP Phase I. Revising baselines and targets based on 
improved data quality will help states to more accurately 
determine the progress of children or youth as it relates 
to their SIMR. 
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Appendix A 


State SIMRs 


Program 
Part C 


SIMRs 

Early Childhood Outcomes 
e Social-Emotional 
e Knowledge and Skills 
e Behavior Meets Needs 


Family Outcomes 
e Develop and Learn 


e Effectively Communicate 
Children’s Needs and Help 
Children Develop and 
Learn 


Other 


Part B 
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e Graduation 

Reading/ELA 

e Math 

e Reading and Math 

Early Childhood Outcomes 
Post-School Outcomes 
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